The Essential Role of Open Data and Software for the Future of Ultrasound-Based Neuronavigation

With the recent developments in machine learning and modern graphics processing units (GPUs), there is a marked shift in the way intra-operative ultrasound (iUS) images can be processed and presented during surgery. Real-time processing of images to highlight important anatomical structures combined with in-situ display, has the potential to greatly facilitate the acquisition and interpretation of iUS images when guiding an operation. In order to take full advantage of the recent advances in machine learning, large amounts of high-quality annotated training data are necessary to develop and validate the algorithms. To ensure efficient collection of a sufficient number of patient images and external validity of the models, training data should be collected at several centers by different neurosurgeons, and stored in a standard format directly compatible with the most commonly used machine learning toolkits and libraries. In this paper, we argue that such effort to collect and organize large-scale multi-center datasets should be based on common open source software and databases. We first describe the development of existing open-source ultrasound based neuronavigation systems and how these systems have contributed to enhanced neurosurgical guidance over the last 15 years. We review the impact of the large number of projects worldwide that have benefited from the publicly available datasets “Brain Images of Tumors for Evaluation” (BITE) and “Retrospective evaluation of Cerebral Tumors” (RESECT) that include MR and US data from brain tumor cases. We also describe the need for continuous data collection and how this effort can be organized through the use of a well-adapted and user-friendly open-source software platform that integrates both continually improved guidance and automated data collection functionalities.publishedVersio

How well do neurosurgeons predict survival in patients with high-grade glioma?

Due to the lack of reliable prognostic tools, prognostication and surgical decisions largely rely on the neurosurgeons’ clinical prediction skills. The aim of this study was to assess the accuracy of neurosurgeons’ prediction of survival in patients with high-grade glioma and explore factors possibly associated with accurate predictions. In a prospective single-center study, 199 patients who underwent surgery for high-grade glioma were included. After surgery, the operating surgeon predicted the patient’s survival using an ordinal prediction scale. A survival curve was used to visualize actual survival in groups based on this scale, and the accuracy of clinical prediction was assessed by comparing predicted and actual survival. To investigate factors possibly associated with accurate estimation, a binary logistic regression analysis was performed. The surgeons were able to diferentiate between patients with diferent lengths of survival, and median survival fell within the predicted range in all groups with predicted survival24 months, median survival was shorter than predicted. The overall accuracy of surgeons’ survival estimates was 41%, and over- and underestimations were done in 34% and 26%, respectively. Consultants were 3.4 times more likely to accurately predict survival compared to residents (p=0.006). Our fndings demonstrate that although especially experienced neurosurgeons have rather good predictive abilities when estimating survival in patients with high-grade glioma on the group level, they often miss on the individual level. Future prognostic tools should aim to beat the presented clinical prediction skills.publishedVersio

Meningioma segmentation in T1-weighted MRI leveraging global context and attention mechanisms

Meningiomas are the most common type of primary brain tumor, accounting for approximately 30% of all brain tumors. A substantial number of these tumors are never surgically removed but rather monitored over time. Automatic and precise meningioma segmentation is therefore beneficial to enable reliable growth estimation and patient-specific treatment planning. In this study, we propose the inclusion of attention mechanisms over a U-Net architecture: (i) Attention-gated U-Net (AGUNet) and (ii) Dual Attention U-Net (DAUNet), using a 3D MRI volume as input. Attention has the potential to leverage the global context and identify features' relationships across the entire volume. To limit spatial resolution degradation and loss of detail inherent to encoder-decoder architectures, we studied the impact of multi-scale input and deep supervision components. The proposed architectures are trainable end-to-end and each concept can be seamlessly disabled for ablation studies. The validation studies were performed using a 5-fold cross validation over 600 T1-weighted MRI volumes from St. Olavs University Hospital, Trondheim, Norway. For the best performing architecture, an average Dice score of 81.6% was reached for an F1-score of 95.6%. With an almost perfect precision of 98%, meningiomas smaller than 3ml were occasionally missed hence reaching an overall recall of 93%. Leveraging global context from a 3D MRI volume provided the best performances, even if the native volume resolution could not be processed directly. Overall, near-perfect detection was achieved for meningiomas larger than 3ml which is relevant for clinical use. In the future, the use of multi-scale designs and refinement networks should be further investigated to improve the performance. A larger number of cases with meningiomas below 3ml might also be needed to improve the performance for the smallest tumors.Comment: 16 pages, 5 figures, 3 tables. Submitted to Artificial Intelligence in Medicin

FastPathology: An open-source platform for deep learning-based research and decision support in digital pathology

Deep convolutional neural networks (CNNs) are the current state-of-the-art for digital analysis of histopathological images. The large size of whole-slide microscopy images (WSIs) requires advanced memory handling to read, display and process these images. There are several open-source platforms for working with WSIs, but few support deployment of CNN models. These applications use third-party solutions for inference, making them less user-friendly and unsuitable for high-performance image analysis. To make deployment of CNNs user-friendly and feasible on low-end machines, we have developed a new platform, FastPathology, using the FAST framework and C++. It minimizes memory usage for reading and processing WSIs, deployment of CNN models, and real-time interactive visualization of results. Runtime experiments were conducted on four different use cases, using different architectures, inference engines, hardware configurations and operating systems. Memory usage for reading, visualizing, zooming and panning a WSI were measured, using FastPathology and three existing platforms. FastPathology performed similarly in terms of memory to the other C++ based application, while using considerably less than the two Java-based platforms. The choice of neural network model, inference engine, hardware and processors influenced runtime considerably. Thus, FastPathology includes all steps needed for efficient visualization and processing of WSIs in a single application, including inference of CNNs with real-time display of the results. Source code, binary releases and test data can be found online on GitHub at https://github.com/SINTEFMedtek/FAST-Pathology/.Comment: 12 pages, 4 figures, submitted to IEEE Acces

Survival of glioblastoma in relation to tumor location: a statistical tumor atlas of a population-based cohort

Purpose: Previous studies on the effect of tumor location on overall survival in glioblastoma have found conflicting results. Based on statistical maps, we sought to explore the effect of tumor location on overall survival in a population-based cohort of patients with glioblastoma and IDH wild-type astrocytoma WHO grade II–III with radiological necrosis. Methods: Patients were divided into three groups based on overall survival: 24 months. Statistical maps exploring differences in tumor location between these three groups were calculated from pre-treatment magnetic resonance imaging scans. Based on the results, multivariable Cox regression analyses were performed to explore the possible independent effect of centrally located tumors compared to known prognostic factors by use of distance from center of the third ventricle to contrast-enhancing tumor border in centimeters as a continuous variable. Results: A total of 215 patients were included in the statistical maps. Central tumor location (corpus callosum, basal ganglia) was associated with overall survival 24 months. Increased distance from center of the third ventricle to contrast-enhancing tumor border was a positive prognostic factor for survival in elderly patients, but less so in younger patients. Conclusions: Central tumor location was associated with worse prognosis. Distance from center of the third ventricle to contrast-enhancing tumor border may be a pragmatic prognostic factor in elderly patients.publishedVersio

Code-Free Development and Deployment of Deep Segmentation Models for Digital Pathology

Application of deep learning on histopathological whole slide images (WSIs) holds promise of improving diagnostic efficiency and reproducibility but is largely dependent on the ability to write computer code or purchase commercial solutions. We present a code-free pipeline utilizing free-to-use, open-source software (QuPath, DeepMIB, and FastPathology) for creating and deploying deep learning-based segmentation models for computational pathology. We demonstrate the pipeline on a use case of separating epithelium from stroma in colonic mucosa. A dataset of 251 annotated WSIs, comprising 140 hematoxylin-eosin (HE)-stained and 111 CD3 immunostained colon biopsy WSIs, were developed through active learning using the pipeline. On a hold-out test set of 36 HE and 21 CD3-stained WSIs a mean intersection over union score of 95.5 and 95.3% was achieved on epithelium segmentation. We demonstrate pathologist-level segmentation accuracy and clinical acceptable runtime performance and show that pathologists without programming experience can create near state-of-the-art segmentation solutions for histopathological WSIs using only free-to-use software. The study further demonstrates the strength of open-source solutions in its ability to create generalizable, open pipelines, of which trained models and predictions can seamlessly be exported in open formats and thereby used in external solutions. All scripts, trained models, a video tutorial, and the full dataset of 251 WSIs with ~31 k epithelium annotations are made openly available at to accelerate research in the field.Peer reviewe

RESECT-SEG: Open access annotations of intra-operative brain tumor ultrasound images

Purpose: Registration and segmentation of magnetic resonance (MR) and ultrasound (US) images play an essential role in surgical planning and resection of brain tumors. However, validating these techniques is challenging due to the scarcity of publicly accessible sources with high-quality ground truth information. To this end, we propose a unique annotation dataset of tumor tissues and resection cavities from the previously published RESECT dataset (Xiao et al. 2017) to encourage a more rigorous assessments of image processing techniques. Acquisition and validation methods: The RESECT database consists of MR and intraoperative US (iUS) images of 23 patients who underwent resection surgeries. The proposed dataset contains tumor tissues and resection cavity annotations of the iUS images. The quality of annotations were validated by two highly experienced neurosurgeons through several assessment criteria. Data format and availability: Annotations of tumor tissues and resection cavities are provided in 3D NIFTI formats. Both sets of annotations are accessible online in the \url{https://osf.io/6y4db}. Discussion and potential applications: The proposed database includes tumor tissue and resection cavity annotations from real-world clinical ultrasound brain images to evaluate segmentation and registration methods. These labels could also be used to train deep learning approaches. Eventually, this dataset should further improve the quality of image guidance in neurosurgery.Comment: Bahareh Behboodi and Francois-Xavier Carton share the first authorshi

Brain-shift compensation using intraoperative ultrasound and constraint-based biomechanical simulation

International audiencePurpose. During brain tumor surgery, planning and guidance are based on pre-operative images which do not account for brain-shift. However, this deformation is a major source of error in image-guided neurosurgery and affects the accuracy of the procedure. In this paper, we present a constraint-based biome-chanical simulation method to compensate for craniotomy-induced brain-shift that integrates the deformations of the blood vessels and cortical surface, using a single intraoperative ultrasound acquisition. Methods. Prior to surgery, a patient-specific biomechanical model is built from preoperative images, accounting for the vascular tree in the tumor region and brain soft tissues. Intraoperatively, a navigated ultrasound acquisition is performed directly in contact with the organ. Doppler and B-mode images are recorded simultaneously, enabling the extraction of the blood vessels and probe footprint respectively. A constraint-based simulation is then executed to register the pre-and intraoperative vascular trees as well as the cortical surface with the probe footprint. Finally, preoperative images are updated to provide the surgeon with images corresponding to the current brain shape for navigation. Results. The robustness of our method is first assessed using sparse and noisy synthetic data. In addition, quantitative results for five clinical cases are provided , first using landmarks set on blood vessels, then based on anatomical structures delineated in medical images. The average distances between paired vessels landmarks ranged from 3.51 to 7.32 (in mm) before compensation. With our method, on average 67% of the brain-shift is corrected (range [1.26; 2.33]) against 57% using one of the closest existing works (range [1.71; 2.84]). Finally, our method is proven to be fully compatible with a surgical workflow in terms of execution times and user interactions. Conclusion. In this paper, a new constraint-based biomechanical simulation method is proposed to compensate for craniotomy-induced brain-shift. While being efficient to correct this deformation, the method is fully integrable in a clinical process

Vessel-based brain-shift compensation using elastic registration driven by a patient-specific finite element model

International audienceDuring brain tumor surgery, planning and guidance are based on pre-operative images which do not account for brain-shift.However, this shift is a major source of error in neuro-navigation systems and affects the accuracy of the procedure. The vascular tree is extracted from pre-operative Magnetic Resonance Angiography and from intra-operative Doppler ultrasound images, which provides sparse information on brain deformations.The pre-operative images are then updated based on an elastic registration of the blood vessels, driven by a patient-specific biomechanical model.This biomechanical model is used to extrapolate the deformation to the surrounding soft tissues.Quantitative results on a single surgical case are provided, with an evaluation of the execution time for each processing step.Our method is proved to be efficient to compensate for brain deformation while being compatible with a surgical process